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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
September 6, 2007 has been entered. 

2. Claims 1 and 3-10 are pending in the application. Applicant has cancelled claims 
2 and 11-15. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.SC 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 
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5. Claims 1, 3, 6, 8, 9 and 10 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Nakamura et al.(US 6,963,374) in view of Anderson(US 6,847,388). 

Consider claim 1 , Nakamura et al. teach: 

An image sensing apparatus("Digital Camera", figures 1-4, column 2, line 56 
through column 4, line 43) comprising: 

an image sensing device("CCD", 303, figure 4) which outputs image data 
obtained by an image sensing element(column 3, lines 50-58); 

a white balance integration device("Black Level Correction/WB", 21 1a, figure 6a) 
which integrates the image data output from said image sensing device for white 
balance processing(column 5, lines 55-63); 

an image processing device(21 1 , figure 4) which performs a color space 
conversion processing for the image data obtained by said image sensing 
device(column 4, lines 8-10, lines 55-58);. 

a display device("EVF", 20, or "LCD", 10;- figure 4) which displays an object 
image during imaging on the image sensing element(The display acts as a "live view 
display"(i.e. an object image is displayed during imaging), column 3, lines 16-23.); and 

a control devicefmain CPU", 21 , figure 4) which controls said white balance • 
integration device(211a, column 5, lines 50-63), said image processing device(211, 
column 5, lines 50-66), and said display device(20, 10, column 5, line 66 through 
column 6, line 3), so that, said image processing device processes the color space 
conversion for second image data obtained by previous image sensing operation(See 
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"Pc", figure 8, column 7, lines 24-33, step S10, figure 7) in accordance with the start of 
the exposure/storage of first image data from the image sensing element(column 7, 
lines 24-33), the integral processing for the first image data(See "Pe", figure 8) by said 
white balance integration device(column 7, lines 41-49, step ST12, figure 7) and the 
color space conversion("Pc", figure 8) for second image data by said image processing 
device(21 1 ) processes are performed in series during read of the first image signal from 
the image sensing element(See figure 8, column 7, lines 24-49), and said display 
device("LCD", 1 0) displays the obiect image after the color space conversion("Pc") 
processing for the second image data ends(see figure 8) in case the integral processing 
for the first image data ends earlier than the color space conversion processing for the 
second image data ends(See figure 8. With regards to Nakamura et al., figure 8 shows 
that the color space conversion processing("Pc") of the second image data ends earlier 
than the integral processing of the first image data("Pe"), and makes no reference to the 
integral processing ending first. However, this point is discussed with respect to 
Anderson, see below.). 

However, Nakamura et al. do not explicitly teach that said image processing 
device processes the color space conversion for second image data in accordance with 
the start of reading of the first image data from the image sensing element. Nakamura 
et al. do not explicitly teach that integral processing of the first image data and the color 
space conversion of the second image data are performed in parallel . 

Anderson is similar to Nakamura et al. in that Anderson teaches of a 
camera(figures 1-3) with a memory(figure 4a). Anderson also similarly teaches of 
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reading out raw image data from an image sensor(1 14, figure 1 , column 5, lines 59-64), 
storing the data in a memory(530, column 5, line 59 through column 6, line 3), and 
subsequently performing color space conversion on the image data(column 8, line 59 
through column 9, line 7). 

However, in addition to the teachings of Nakamura et al., Anderson teaches that 
the image processing device processes the color space conversion for second image 
data in accordance with the start of reading of the first image data from the image 
sensing element(See figure 4a, column 4, line 59 through column 6, line 3, column 6, 
lines 38-56, column 8, line 59 through column 9, line 8. Anderson teaches, "Referring 
again to FIG. 4B, the ping-pong buffers are utilized during live view mode as follows. 
While input buffer A is filled with image data, the data from input buffer B is processed 
and transmitted to frame buffer B. At the same time, previously processed data in 
frame buffer A is output to the LCD screen 402 for display. While input buffer B is filled 
with image data, the data from input buffer A is processed and transmitted to frame 
buffer A. At the same time, previously processed data in frame buffer B is output to the 
LCD screen 402 for display." As one buffer is filled with raw image data, the other 
buffer is emptied and processed, which processing involves color space 
conversion(See 612, figure 7, column 8, line 59 through column 9, line 8).). - 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use ping-pong buffers as taught by Anderson in the 
camera taught by Nakamura et al. to read out raw image data from the image sensor 
concurrent with the processing color space conversion of previous image data, for the 
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benefit of improving the display speed of the digital camera and preventing the tearing 
of the image on the display(Anderson, column 5, line 65 through column 6, line 3.). 

Because the white balance integration taught by Nakamura et al. is performed 
during the raw data writing("Pe"), and thus in parallel with the readout of the first image 
data(see figure 8), the combination of Nakamura et al. and Anderson teaches that that 
integral processing of the first image data and the color space conversion of the second 
image data are performed in parallel. 

Also, because ping-pong buffers are used in Anderson, and the color space 
conversion(see 612, figure 7) is performed concurrent with the readout/integral 
processing of the image sensor(see above), yet prior to display(see figure 7), the 
display device displays the obiect image after the color space conversion processing for 
the second image data ends(see figure 7, Anderson) regardless of whether the integral 
processing for the first image data ends earlier than the color space conversion 
processing for the second image data ends. 

Consider claim 3, and as applied to claim 1 above, Nakamura et al. further teach: 
the first image data includes image data having a signal amount corresponding to 

a color filter of the image sensing element(column 3, lines 50-58, the image signal is 

filtered into red, green, and blue image data), 

and the second image data includes image data capable of confirming the object 

image(column 4, lines 11-19). 
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Consider claim 6, and as applied to claim 1 above, Nakamura et al. teach that a 
control device is used to control all functions of the camera(see claim 1 rationale) and 
that integral processing is performed on the image data after the display device displays 
the object image during imaging on the image sensing element(see claim 1 rationale). 

However, Nakamura et al. do not explicitly teach the apparatus further comprises 
a thumbnail image generation device, which generates a thumbnail image on the basis 
of the first image data. 

However, Anderson teaches that the apparatus further comprises a thumbnail 
image generation device which generates a thumbnail image on the basis of the first 
image data(See 606 and 616, figures 6 and 7, column 9, lines 9-15. Anderson teaches 
that the raw data used for live view display( i.e. first image data) is used to create a 
thumbnail image, due to the fact that the thumbnail image need not be a high resolution 
image.). 

Anderson further teaches that said control device controls said thumbnail image 
generation device so as to generate the thumbnail image after causing said display 
device to display the object image during imaging on the image sensing 
element(column 9, lines 16-40). 

Consider claim 8, and as applied to claim 1 above, Nakamura et al. further teach 
of a temporary storage device("DRAM", 232, and "Memory Card", 8, figure 4) which 
temporarily stores at least two first image data(DRAM(232) reads in raw data(first first 
image data) over channel 1, and reads out preceding image data(second first image 
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data) over channel 2, column 7, lines 15-49. Therefore, both first image data are stored 
in DRAM temporarily.) and one second image data(The memory card(8) stores second 
image data(i.e. processed image data), column 7, lines 34-38). The Ping-Pong buffers 
of Anderson(figure 4a) also work in this fashion. 

Consider claim 9, and as applied to claim 1 above, Nakamura et al. further teach: 
Said control device so controls as to start processing of said image processing 
device(21 1a) at any one of a timing at which a photographing instruction switch is 
released(See figure 7, in step ST1 a shutter release button(i.e. photographing 
instruction switch) is pressed, and this leads to step ST9 wherein white balance 
processing occurs. See column 6, line 57 through column 7, line 23). 

Consider claim 10, and as applied to claim 1 above, Nakamura et al. further 

teach: 

When display operation of said display device stops("live view display is not 
produced", column 7, lines 18-19), said control device so controls as to start processing 
of said image processing device(21 1a) at any timing at which a photographing 
instruction switch is released(See figure 7, in step ST1 a shutter release button(i.e. 
photographing instruction switch) is pressed, and this leads to step ST9 wherein white 
balance processing occurs. A live view display is not produced during step ST9. See 
column 6, line 57 through column 7, line 23). 
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6. Claims 4, 5 and 7 are rejected under 35 (JSC 103(a) as being unpatentable 
over Nakamura et al. in view of Anderson as applied to claim 1 above, and further in 
view of Kagle et al.(US Patent 6,967,680). 

Consider claim 4, and as applied to claim 1 above, Nakamura et al. teach of a 
control device that causes integral processing of first image data after causing a display 
device to display the object image during imaging on the image sensing element(see 
claim 1 rationale). 

However, the combination of Nakamura et al. and Anderson does not explicitly 
teach that the apparatus further comprises a defect correction device which corrects a 
defective pixel portion of image data when the image sensing element has a defective 
pixel, and said control device controls said defect correction device so as to correct a 
defective pixel portion of the first image data during the integral processing. 

Kagle et al. is similar to Nakamura et al. in image data is collected from the 
image sensor, preliminary processing is performed to yield first image data, second 
image data is obtained through post processing, and the final image is stored in 
memory(see figure 2, column 3, line 4 through column 4, line 12). Kagle et al. is also 
similar to Nakamura et al. in that white balance processing is performed during pre- 
processing(column 3, lines 18-23, figure 3). 

However, in addition to the combined teachings of Nakamura et al. and 
Anderson, Kagle et al. teach that the apparatus further comprises a defect correction 
device(428, figure 3) which corrects a defective pixel portion of image data when the 
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image sensing element has a defective pixel(column 3, lines 31-37), and said defect 
correction device(428) corrects a defective pixel portion of the first image data during 
the integral processing(See column 3, lines 31-33, defective pixel processing is 
performed during pre-capture process control(i.e. integral processing), step 304, figure 
2.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have a defect correction device for correcting defective 
pixels as taught by Kagle et al. in the integral image processing on the first image data 
within the camera device taught by the combination of Nakamura et al. and Anderson 
for the benefit that the defect correction device could determine malfunctioning 
pixels(column 3, lines 35-37) and thereby modify the performance characteristics of the 
camera in order to correct for defective pixels in advance, thus minimizing any 
processing delays that are undesirable when taking photographs in rapid 
succession(Kagle et al., column 1, lines 20-47). 

Consider claim 5, and as applied to claim 1 above, Nakamura et al. teach of a 
control device that causes integral processing of first image data after causing a display 
device to display the object image during imaging on the image sensing element(see 
claim 1 rationale). Nakamura et al. also teach of performing white balance processing 
after causing said display device to display the object image during imaging on the 
image sensing element(see claim 1 rationale), and that white balance processing is 
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performed on the basis of an integral result of said white balance integration 
device(column 7, lines 15-19). 

However, the combination of Nakamura et al. and Anderson do not explicitly 
teach that the apparatus further comprises a defect correction device which corrects a 
defective pixel portion of the first image data when the image sensing element has a 
defective pixel, or that a white balance coefficient calculation device calculates a white 
balance coefficient during integral processing to be used to perform white balance 
image processing. 

Kagle et al. is similar to Nakamura et al. in image data is collected from the 
image sensor, preliminary processing is performed to yield first image data, second 
image data is obtained through post processing, and the final image is stored in 
memory(see figure 2, column 3, line 4 through column 4, line 12). Kagle et al. is also 
similar to Nakamura et al. in that white balance processing is performed during pre- 
processing(column 3, lines 18-23, figure 3). 

However, in addition to the teachings of Nakamura et al. and Anderson, Kagle et 
al. teach that the apparatus further comprises a defect correction device(428, figure 3) 
which corrects a defective pixel portion of image data when the image sensing element 
has a defective pixel(column 3, lines 31-37), and said defect correction device(428) 
corrects a defective pixel portion of the first image data during the integral 
processing(See column 3, lines 31-33, defective pixel processing is performed during 
pre-capture process control(i.e. integral processing), step 304, figure 2.). Kagle et al. 
also teach that a white balance coefficient calculation device(404, figure 3) calculates a 
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white balance coefficient(After white balance process control is completed, a processing 
value(i.e. white balance coefficient) is returned to the pre-capture process control, 
column 3, lines 18-27.) during integral processing to be used as a basis to perform 
white balance image processing(The processing results of a first frame of image data 
obtained during pre-processing(i.e. integral processing) are used to process the second 
frame of image data as long as the results are within a threshold. See figure 6, column 
5, lines 23-44). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have a defect correction device for correcting defective 
pixels, and a white balance coefficient calculation device as taught by Kagle et al. in the 
integral image processing on the first image data within the camera device taught by the 
combination of Nakamura et al. and Anderson for the benefit that the defect correction 
device could determine malfunctioning pixels(column 3, lines 35-37), and the white 
balance coefficient calculation device could produce values which could be applied to 
later image frames, thereby enabling the camera to modify the performance 
characteristics in order to correct for defective pixels in advance, and skip the step of 
determining a white balance coefficient when unnecessary, thus minimizing any 
processing delays that are undesirable when taking photographs in rapid 
succession(Kagle et al., column 1, lines 20-47). 

Consider claim 7, and as applied to claim 1 above, Nakamura et al. teach of a 
control device and a display device(see claim 1 rationale). 
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However, the combination of Nakamura et al. and Anderson does not explicitly 
teach that the apparatus further comprises a defect correction device which corrects a 
defective pixel portion of image data when the image sensing element has a defective 
pixel, and said control device controls said defect correction device so as to correct a 
defective pixel portion of the first image data before causing said image processing 
device to start the image processing after causing said display device to display the 
object image. 

Kagle et al. is similar to Nakamura et al. in image data is collected from the 
image sensor, preliminary processing is performed to yield first image data, second 
image data is obtained through post processing, and the final image is stored in 
memory(see figure 2, column 3, line 4 through column 4, line 12). Kagle et al. is also 
similar to Nakamura et al. in that white balance processing is performed during pre- 
processing(column 3, lines 18-23, figure 3). 

However, in addition to the combined teachings of Nakamura et al. and 
Anderson, Kagle et al. teach that the apparatus further comprises a defect correction 
device(428, figure 3) which corrects a defective pixel portion of image data when the 
image sensing element has a defective pixel(column 3, lines 31-37), and said defect 
correction device(428) corrects a defective pixel portion of the first image data during 
the integral processing(See column 3, lines 31-33, defective pixel processing is 
performed during pre-capture process control(i.e. integral processing, before the start of 
image processing), step 304, figure 2.). 



Application/Control Number: 10/702,200 Page 14 

Art Unit: 2622 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have a defect correction device for correcting defective 
pixels as taught by Kagle et al. in the integral image processing on the first image data 
within the camera device taught by the combination of Nakamura et al. and Anderson 
for the benefit that the defect correction device could determine malfunctioning 
pixels(column 3, lines 35-37) and thereby modify the performance characteristics of the 
camera in order to correct for defective pixels in advance, thus minimizing any 
processing delays that are undesirable when taking photographs in rapid 
succession( Kagle et al., column 1, lines 20-47). 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Albert H. Cutler whose telephone number is (571 )-270- 
1460. The examiner can normally be reached on Mon-Fri (7:30-5:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ngoc-Yen Vu can be reached on (571)-272-7320. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
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USPTO Customer Service Representative or access to the automated information 
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